1. Technical Field
The present invention relates generally to an improved data processing system. More particularly, the present invention relates to a mechanism for an efficient implementation of a shared receive queue.
2. Description of Related Art
InfiniBand and remote direct memory access (RDMA) protocols provide a message passing mechanism which can be used for Input/Output (I/O) devices and interprocess communications between computing nodes. Consumers access Infiniband message passing hardware by posting send and receive messages to respective send and receive work queues (WQs) on an Infiniband channel adapter. The send/receive work queues are assigned to a consumer as a queue pair. Consumers retrieve the results of these messages from a completion queue (CQ) through the Infiniband send and receive work completions.
RDMA extends the Infiniband semantics to allow sharing a single receive queue between several connections. A shared receive queue (SRQ) allows sharing of buffer resources between multiple connections and facilitates minimizing the overall size of the buffers needed to be posted to sustain a particular throughput.
In non-shared receive queue (RQ) implementations, posted work requests (WRs) are completed in the order that they are posted. This allows efficient management of an RQ by software and fetching of multiple WQEs using a single DMA operation by the adapter. In SRQ implementations, on the other hand, WRs are not necessarily completed in order they have been posted. Different WQEs are consumed by different connections, and connections may vary by their respective reception rate and the message size. Consequently, work requests may be completed out of order. A result of such an implementation is that an SRQ location can be consumed by a non-completed WQE due to the our-of-order WQE completion—an undesirable effect referred to as “head-of-line blocking.”
To alleviate head-of-line blocking, a linked list approach may be implemented. In this approach, each WQE points to the next WQE in the SRQ. WQEs are added to the tail of the list and can be removed from the middle of the list. However, this approach leads to the discontinuous placement of WQEs in the memory and prevents efficient multiple WQE fetches by the adapter.
It would be advantageous to provide an improved mechanism for posting and retrieving WQEs to a shared receive queue. It would be further advantageous to provide a mechanism for allowing non-sequential entry postings in an SRQ. It would be further advantageous to provide a mechanism for non-sequentially retrieving WQEs from an SRQ.